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The invention relates to trick play behavior of digital video data, and more 
particularly to a method and apparatus for controlh'ng flie operation of an application 
associated v^ith the digital video data during trick play operation. 



A broadcaster can broadcast multimedia platform-specific applications 
together with digital television programs, with a suitably equipped platform-specij&c set-top 
box receiving those applications and running them locally. Examples of such applications are 
electronic program guides, play-along games, tele-banking, tele-shopping, electronic 
newspapers and similar information services. Television programs can be recorded and, if 
sudb-a^television program has an application associated with it, then that application should 
.also J^j-ecorded. Typically multimedia platform-specific applications are broadcast in a data 
or object carousel, where all the application code and data is broadcast in cycles. This 
resembles teletext data, which is also broadcast in a carousel. 

A suitable transmission system for such application delivery is known from 
ISO/IEC hitemational Standard 13818-6, "MPEG-2 Digital storage Media Command and 
Control" July 12, 1996. In modem digital broadcast systems a transmitter typically transmits 
a large number of services (or chaimels) to a plurality or receivers, examples of which are to 
be found in television sets or set-top boxes. Such a service can contain an audio/video stream, 
an interactive application, for example in the MHEG-5 format, other kinds of data or a 
corgbination of these elements. An MPEG-2 transport stream is a multiplex of a number of 
sr-^ni5^, and a transmitter will typically transmit several transport streams to the set-top 
boxes. In turn, a suitably configured set-top box can be tuned to a specific transport stream 
and is then able to retrieve information firom that transport stream. 

When running an application that is retrieved directly firom a Uve broadcast, 
the "^application time" increases linearly. A user cannot jump back and forth in a live 
broadcast. When replaying a recorded application, however, the user can select several trick 
play operations such as fast forward, or (fast) rewind the recording. However, it is often the 
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case that an application that has been written for linear playback can not handle these fast 
forward and fast or conventional speed rewind (or more general, non-linear time) operations. 
Reasons for this include that an application can get events that are broadcast with the 
application, and the non-linear playback could result in missing such events or getting such 
5 events multiple times. An example of such an event is "Player X of football team Y has 
scored a goal." 

When performing trick play over a digital interface, the stream is significantly 
transformed, e.g., only the I-frames are extracted and empty P-frames are added to cause 
repetition of the I-frames. In this case, only parts of the original stream are used for trick play 

10 and so only parts of the interactive applications will be present in the trick play stream. Also 
after trick play, the application may be in an inconsistent slate which may result in problems. 
It will be understood that it is possible to write applications that can work well in the context 
of non-linear time. The problem is that not all applications can be expected to be authored 
such that they do work well in the context of non-linear time. Thus, there is a need for a 

1 5 method and apparatus for controlling the operation of the application during trick play 
operations so that the application is in a consistent state after the trick play operation is 
completed.^. 



.20 It is m object Qf thfi i^vmtiOJaJaJa3^^ the^bjoy-erdescribed-deficiencies 

by providing a method and apparatus for controlling tiie operation of an application during 
trick play operation by selectively inserting parts of the interactive content into the trick play 
stream to ensure that the behavior of the apptication is well defined. According to one 
embodiment of the invention, events are supplied to the application during trick play 

25 operations. 

According to one embodiment of the invention, a method and apparatus for the 
handling of a recorded data stream and associated linear appUcation is disclosed. Linear real- 
time playback of the data stream and running of the linear application is commenced from a 
starting point thereof. On entering a non real-time playback phase, select frames from the 
30 data stream are mapped using a mapping scheme to create an interactive trick pky stream. 
Events from the linear application are mapped into the interactive trick play stream using the 
mapping scheme. 

These and other aspects of the invention will be apparent from and elucidated 
with reference to the embodiments described hereafter. 
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The invention will now be described, by way of example, with reference to the 
accompanying drawings, wherein: 
5 Figure 1 illustrates a block diagram of a audio-video apparatus suitable to host 

embodiments of the invention; 

Figure 2 illustrates a block diagram of a set-top box which can be used to 
implement at least one embodiment of the invention; 

Figures 3(a)-(b) illustrate several trick play processing systems according to 
10 one embodiment of the invention; 

Figure 4 illustrates an original time line and a new time line for fast forward 
operations according to one embodiment of the invention; 

Figure 5 is a flow chart depicting the control of the operation of the interactive 
appUcation during fast forward operations according to one embodiment of the invention. 

15 

Figure 1 illustrates an audio-video apparatus 100 suitable to host the invention. 
The^paratus comprises an input terminal 1 for receiving a digital video signal to be 
recorded on a disc 3. Further, the ^paratus comprises an output terminal 2 for supplying a 

20 digital video signal reproduced from the disc. These terminals may in use be connected via a 
digital interface to a digital television receiver and decoder in the form of a set-top box 
(STB)12, which also receives broadcast signals from satellite, cable or the like, in MPEG TS 
format. The set-top box 12 provides display signals to a display device 14, which may be a 
conventional television set, 

25 The video recording apparatus 100 as shown in Figure 1 is composed of two 

major system parts, namely the disc subsystem 6 and the video recorder subsystem 8, 
controlling both recording and playback. The two subsystems have a number of features, as 
will be readily understood, including that the disc subsystem can be addressed transparently 
in terms of logical addresses (LA) and can guarantee a maximum sustainable bit-rate for 

30 reading and/or writing data from/to the disc. 

Suitable hardware arrangements for implementing such an apparatus are 
known to one skilled in the art, with one example illustrated in patent application WO-A- 
00/00981. The apparatus generally comprises signal processing units, a read/write unit 
including a read/write head configured for reading from/writing to the disc 3. Actuators 
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position the head in a radial direction across the disc, while a motor (not shown) rotates the 
disc 3, A microprocessor (not shown) is present for controlling all the circuits in a known 
manner. 

Referring to Figure 2, a block diagram of the set-top box 12 is shown. It will 
5 be imderstood by those skilled in the art that the invention is not limited to a set top box but 
also extends to a variety of devices such as a DVD player, PVR box, a box containing a Hard 
disk (recorder module), etc, A broadcast signal 39 is received and fed into a tuner 31, The 
tuner 3 1 selects the channel on which the broadcast audio-video-interactive signal is 
transmitted and passes the signal to a processing unit 32. The processing imit 32 

10 demultiplexes the packets from the broadcast signal if necessary and reconstructs the 

television programs and/or interactive applications embodied in the signal. The programs and 
applications are then decompressed by a decompression unit 33. The audio and video 
information associated with the television programs embodied in the signal is then conveyed 
to a display unit 34, which may perform further processing and conversion of the infoimation 

1 5 into a suitable television format, such as NTSC or HDTV audio/video. Applications 

reconstructed Irom the broadcast signal are routed to random access memory (RAM) 37 and 
are executed by a control system 35 . . 

. The control system 35 may include a microprocessor, micro-controller, digital 
signal processor (DSP), or some other type of sojEtware instruction processing device, and 

static (e.g. SRAM), dynamic (e.g. DRAM), volatile or non-volatile (e.g., FLASH), as 
required to support the functions of the set-top box. When power is applied to the set-top box, 
the control system 35 executes operating system code which is stored in ROM 36. The 
operating system code executes continuously while the set-top box is powered in the same 

25 manner as the operating system code of a typical personal computer and enables the set-top 
box to act on control information and execute interactive and other applications. The set-top 
box also includes a modem 38. The modem 38 provides both a return path by which viewer 
data can be transmitted to the broadcast station and an alternate path 41 by which the 
broadcast station can transmit data to the set-top box, 

30 Although the term "set-top box" is used herein, it will be understood that this 

term refers to any receiver or processing unit for receiving and processing a transmitted 
signal and conveying the processed signal to a television or other monitor. The set-top box 
may be in a housing which physically sits on top of a television, it may be in some other 
location from the television, or it may be incorporated into the television itself. 
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According to one embodiment of the invention, parts of the interactive content 
are selectively supplied to the application to ensure that the behavior of the interactive 
application is well defined and appears consistent and predictable to the user. Two situations 
are to be considered. The first is where the AA^ content and interactive content (iTV 
5 applications and events) are stored together in the same way as it is in the broadcast, as 
illustrated in Figure 3(a). In this example, the AA^ content and the interactive content are 
included in a single stream 302, which is supplied to the trick play processor 304. The trick 
play processor 304 outputs the trick play stream 310. The second case is where the interactive 
content is separated fi-om the AfV content and stored separately, as illustrated in Figure 3(b). 

10 In this example, parts of the original A/V stream are provided in stream 306 to the trick play 
processor 304 and the interactive content is provided to the trick play processor 304 in stream 
308. The trick play processor outputs the trick play stream 3 12. In both cases, the trick play 
processing is essentially the same, however, the latter case will require much less data to be 
retrieved in order to generate the interactive trick play streams. It will be imderstood by those 

15 skilled in the art that the trick play processor 304 can be part of the control system 35 or a 
separate xmit connected to the control system 35. 

The AA^ content transmitted over the digital int^ace shoiild be a valid 
MPEG-2 transport stream in the case of trick play. Similarly, in the case of trick play with 
interactivity, the content transmitted must be a valid stream. To generate a valid iTV stream 

20 both applications and stream events are inserted into a MPEG-2 multiplexer. 

The discussion of the invention will now be divided into several sections each 
dealing with a different trick play operation. For fast forward operations, the original time 
line is mapped onto a new time line as depicted in Figure 4. Events that occur in the original 
stream must also be mapped onto the fast forward stream according to the same mapping. If 

25 the fast forward factor is n and the time trick play is started is t(/o), then event Ex in the 

original stream that occurs at time t(E;^ should be mapped to the fast forward stream at time 

t(ExO = t(Io) + (t(Ex)-t(Io))/n 
Altematively, if Ex occurs between ly and ly+i in the original stream then Ex should again 
occur between ly and ly+i in the fast forward stream. Greater accuracy than this is not really 

30 necessary. Similarly, if the version of the application (module in MHP) changes at a 

particular time in the original stream, then it should be remultiplexed so that the new module 
is loaded at the corresponding time in the trick play stream. 

One potential problem with this solution is that the set-top box may prompt the 
user for some input or display some information in response to an event and wait for a 
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predetermined period of time. In the fast forward case, the next event may be received before 
the predetemiined waiting period of the last event has expired. This might result in undefined 
behavior from the application. To ensure that the application is in a well defined state before 
each event is received, a Pause event followed by an Unpause event can be inserted into the 
stream before each new event. If the application is waiting for a response jfirom the user to a 
previous event, the PauseAJnpause operation will cause the application to use a default 
response as if the user did not respond to the event witihdn the predetermined period of time. 
This will ensure that the application is no longer waiting for input or is using the display 
when the next event occurs. 

The Pause/Unpause operation can also be used in MHP (multimedia home 
platform). The MHP specification extends the existing DVB open standards for broadcast 
and interactive services in all transmission networks including satellite, cable, terrestrial and 
microwave systems. The MHP specification provides a consistent set of features and 
fimctions required for the enhanced broadcasting and interactive broadcasting profiles, hi 
MHP, controlling the lifecycle of an appHcation can be performed remotely using the AIT 
(Application Information Table), which allows an application control code to be specified. 
By changing the value of this code, the appUcation can be started and with an extension to the 
standard could also be paused and unpaused prior to each event during the trick play 
operation. 

2Q , AnJJilustratiKejexampleLOdEthe-in^centl^^ 

appKcation during a fast forward operation will now be described with reference to Figure 5. 
First, the play back of the recorded material and the linear application are started in step 502. 
When the fast forward operation is started in step 504, the trick play processor 304 
determines when an event will occur for the linear application in step 506. Prior to each 
event, the trick play processor 304 causes the application to pause and then unpause in steps 
508 and 510, respectively. After the appUcation has been unpaused, the event is suppHed to 
the application for execution in step 512. 

During fast or slow reverse, there are two possibilities. The first option is to 
send no iTV content to the set-top box during rewind and subsequent play until the point 
where the rewind started is reached. In this case anything displayed by the application will 
remain on the screen during the rewind but no inconsistencies will occur in the application. 
The second option is to pause the application when the user starts rewindmg. Then when the 
playback reaches the point where the rewind started, the appUcation will be unpaused. This 
ensures that during rewind nothing is displayed on the screen. 
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Slow forward is realized by including extra dummy frames in the content to 
cause the repetition of the original frames. In the case where iTV content is stored along with 
the AA^ content, there is no particidar action to take. The iTV applications and events are 
already in the stream and adding dummy frames does not change the relative positions of the 
5 iTV events and applications. When the iTV content is stored along with the AA^ content, the 
applications and events must be inserted in the slow motion stream. There is a direct mapping 
between the time of the original and the time of the slowed down stream so it is straight 
forward to calculate where to insert the events/applications. Alternatively, they can be 
inserted in the stream along with the same AA^ content that they would occur witii in tbe 
1 0 original stream. 

It will be understood that the different embodiments of the invention are not 
Ll u iilt^d to the exact order of the above-described steps as the timing of some steps can be 
interchanged without affecting the overall operation of the invention. Furthermore, the term 
"comprising" does not exclude other elements or steps, the terms "a" and "an" do not exclude 
15 a plurality and a single processor or other unit may ftilfiU the functions of several of the units 
or circuits recited in tite claims. 

The inv^ation may be summarized as follows: 

A method and apparatus for the handling of a recorded data stream and 
associated linear application is disclosed. Linear real-time playback of the data stream and 
20 ruiming of the linear application is commenced from a starting point thereof. On entering a 
non real-time playback phase, select frames from the data stream are mapped using a 
mapping scheme to create an interactive trick play stream. Events from the linear application 
are mapped into the interactive trick play stream using the disclosed mapping scheme. 
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CLAIMS: 

.1 a 10. 2002 



1 . A method for the handling of a recorded data stream and associated linear 

application, comprising the steps of: 

commencing linear real-time playback of the data stream and commencing 
running of the linear application from a starting point thereof; 
5 on entering a non real-time playback phase, mapping select frames from the 

data stream using a mapping scheme to create an interactive trick play stream; and 

mapping events from the linear application into the interactive trick play 
stream using said mapping scheme. 

10 2. The method according to claim 1 , wherein if the event occurs between a first 

and second frame in the recorded data stream, the event is mapped so as to occur between the 
mapped first and second frame in the interactive trick play stream. 

3, The method according to claim 1 , wherein the event is mapped using the 
1 5 following mapping scheme 

t(ExO = t(Io) + (t(Ex)-t(Io))/n 
where n is a fast forward factor, t(Io) is the time trick play is started and Ex is the event, 

4, The method according to claim 1, wherein any event scheduled to occur in a 
20 group-of-pictures is fed to the linear application together with a mapped 1-frame during the 

non real-time feedback. 

5, The method according to claim 1, wherein the event of the linear appUcation is 
executed using no user input 



6. The method according to claim 1, further comprising the steps of: 

determining when an event for the linear application will occur; 

pausing the linear application prior to the event; 

unpausing the linear application prior to the event; and 
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executing the event of the linear application. 

7. The method according to claim 6, wherein the linear application is paused and 
xmpaused using application control codes. 

5 

8. The method according to claim 7, wherein the application control codes are 
received from an Application Xnformation Table. 

9. An apparatus for the handling of a recorded data stream and associated linear 
10 application, comprising: 

means for commencing linear real-time playback of the data stream and 
commencing running of the linear application from a starting point thereof; 

means for mapping select frames from the data stream using a mapping 
scheme to create an interactive trick play stream; and 
15 means for mapping events from the linear application into the interactive trick 

play stream using said mapping scheme. 

1 0. The apparatus according to claim 9, wherein if the event occurs between a fibrst 
and second frame in the recorded data stream, the event is mapped so as to occur between the 

-20 mapped_first-and^econdJBtameJnJ:he-interactiy^tr^ 

1 1. The apparatus according to claim 9, wherein the event is mapped using the 
following mapping scheme 

t(Ex') = t(Io) + (t(EO-t(Io))/n 
25 where n is a fast forward factor, t(Io) is the time trick play is started and Ex is the event. 

12. The apparatus according to claim 9, wherein any event scheduled to occur in a 
group-of-pictures is fed to the linear application together with a mapped I-frame during the 
non real-time feedback. 

30 

13. The apparatus according to claim 9, wherein the event of the Linear application 
is executed using no user input. 



14. 



The apparatus according to claim 9, further comprising: 
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means for determining on entering a non linear playback phase when an event for the linear , , 
application will occur; 

means for pausing the linear application prior to the event; 

means for unpausing the linear application prior to the event; and 
5 means for executing the event of the linear application. 

15. The apparatus according to claim 14, wherein the linear application is paused 

and impaused using application control codes. 

10 16. The apparatus according to claim 15, wherein the application control codes are 

received from an Application Information Table. 

17. A method for the handling of a recorded data stream and associated linear 

appUcation, comprising the steps of: 
15 commencing linear real-time playback of the data stream and commencing 

mnning of the linear application from a starting point thereof; 

on entering a non real-time playback phase, mapping select frames from the 
data stream using a mapping scheme to create an interactive trick play stream; and 

pausing and unpausing the linear application using application control codes 
20 received from an Application Information Table. 
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ABSTRACT: 

^ a 10. 2002 



A method and apparatus for the handling of a recorded data stream and 
associated linear application is disclosed. Linear real-time playback of the data stream and 
running of the linear application is commenced from a starting point thereof. On entering a 
non real-time playback phase, select frames from the data stream are mapped using a 
mapping scheme to create an interactive trick play stream. Events from the linear application 
are mapped into the interactive trick play stream using the disclosed mapping scheme. 
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